How do you read a large file efficiently without loading the entire content into memory?

Question

ICSM Computer · Answer

To read a large file efficiently in C# without loading the entire content into memory, you should read it
line-by-line or in chunks using streams like 
StreamReader or FileStream.
Option 1: Line-by-line with StreamReader (best for text files)
using System;
using System.IO;
class Program
{
    static void Main()
    {
        using (StreamReader reader = new StreamReader("largefile.txt"))
        {
            string line;
            while ((line = reader.ReadLine()) != null)
            {
                // Process each line
                Console.WriteLine(line);
            }
        }
    }
}
This method uses minimal memory, making it ideal for log files, CSVs, etc.
Option 2: Read in byte chunks using FileStream (best for binary or structured data)
using System;
using System.IO;
class Program
{
    static void Main()
    {
        byte[] buffer = new byte[4096]; // 4KB buffer
        using (FileStream fs = new FileStream("largefile.dat", FileMode.Open, FileAccess.Read))
        {
            int bytesRead;
            while ((bytesRead = fs.Read(buffer, 0, buffer.Length)) andgt; 0)
            {
                // Process the bytes
                Console.WriteLine($"Read {bytesRead} bytes");
            }
        }
    }
}
This approach is efficient for reading raw or binary data in segments.
Key Benefits
Avoids high memory usage.Scales well with very large files (GBs or more).Works well for streaming scenarios.

interview

How do you read a large file efficiently without loading the entire content into memory?

ICSM Computer

Can you answer this question?

1 Answers

Option 1: Line-by-line with `StreamReader` (best for text files)

Option 2: Read in byte chunks using `FileStream` (best for binary or structured data)

Key Benefits

Liked By